1. Field of the Invention
This invention relates to image processing, and more particularly, to an apparatus and method for combining a video frame and a graphics frame.
2. Description of the Related Art
Microsoft's MSDN library documentation, such as remote desktop protocol including graphics pipeline extension (MS-RDPEGFX), graphics device interface acceleration extensions (MS-RDPEGDI), a basic connectivity and graphics remote specification (MS-RDPBCGR), provides a description of an image remote system. The image remote system may send data through wire, and operate with a compatible client to receive, decode and render the data. In this image remote system, bitmaps may be transferred from a server to a surface on the compatible client, be transferred between surfaces, or be transferred between a surface and a bitmap cache.
As shown in FIG. 1, FIG. 1 shows a diagram of a conventional image remote system 100. In the embodiment, the image remote system 100 includes a server 101, a network 102, a client device 103, at least a peripheral device 204 and a display device 105. Here, the network 102 may be a network compliant with Microsoft RDP (Remote Desktop Protocol) specification, and the client device 103 may be a thin-client Box. The server 101 includes at least a virtual PC 101a which serves as a host computer for at least a client end (such as the client device 103). The server 101 may deliver a set of graphics encoded data containing a destination region (for a graphics frame) to the client device 103 through the network 102 according to a keyboard/mouse event from the peripheral device 104. In addition, the server 101 may deliver a set of video encoded data with a time stamp and a set of mask encoded data to the client device 103 through the network 102. Finally, the client device 103 may reconstruct a complete frame combining the video frame and the graphics frame to be displayed in a display device 105.
According to Microsoft RDP specification, a video frame is required to be displayed in the display device 105 with reference to a corresponding time stamp while a graphics command “FrameEnd” is issued in the client device 103 after a graphics frame is finished decoding. FIG. 2 is an image combining scheme used in a conventional client device. The video images generally operate at 30 frames per second (fps), but its frame rate can be adjusted according to different network bandwidths.
According to FIG. 2, video frames are decoded and stored in different video decoded buffers 201a˜201c; graphics frames are decoded and stored in different graphics buffers (also called “surface buffers”) 202a˜202b in the client device 103. No matter which frame (video frame or graphics frame) is updated, a BitBlit/Blit function is driven to move the whole frame image data from a back buffer 203 (also called “shadow buffer” or “offscreen buffer”) to primary buffer (also called “front buffer”) 204 for display. FIG. 3 is an exemplary diagram showing different frame updates triggered by two time-stamp events and three FrameEnd events in the conventional client device. As to the example of FIG. 3, since there are five frame updates in total, the whole frame data of the back buffer 203 needs to be moved to the primary buffer 204 five times. It is time-consuming and bandwidth-consuming.
The scheme of FIG. 2 is called a single buffer structure. Since the single buffer architecture needs a large amount of memory access, the system performance is dramatically reduced. Another problem with a single buffer structure is a screen tearing. The screen tearing is a visual artifact where information from two or more different frames is shown in a display device with a single screen draw. For high resolution image, there is no enough time to move frame image content from one buffer to the other buffer in vertical retrace interval of display device. A most common solution to prevent screen tearing is to use multiple frames buffering, such as the double-buffering. However, the multiple frames buffering needs to store a completely frame of an image data. It may result in an increased the cost of memory and processing time.